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DETAILED ACTION 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

1 . Claims 1-6, 18, 22, 26, and 34-36 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Kreitman, U.S. Patent No. 5,491,517. 

With regard to claim 1, Kreitman describes applying a transform function to visual 
content to be inserted into an image sequence (see Kreitman, Figs. 2, 4, and col. 7, line 
64 to col. 8, line 9, wherein image 42 of Fig. 4 is transformed to match the perspective 
of the input video frame of Fig. 2 and is inserted into the video frame of Fig. 2); and 
blending the transformed visual content with the image sequence (see Kreitman, col. 6, 
lines 26-37, Fig. 4C, and col. 8, lines 1-9, wherein transformed blending mask 58 is 
utilized to blend the image 42 with the colors of the field on which the image 42 is 
implanted); wherein the transforming step and the blending step together result in 
insertion of the content into the image sequence such that the content appears at a 
target location as if it had been part of the original scene displayed by the image 
sequence (see Kreitman, col. 8, lines 1-9, wherein if players walk on the part of the 
court 32 where the image 42 is implanted, they will appear to walk "over" the implanted 
image); and wherein the transforming step comprises all applied geometric 
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transformations of the visual content, is performed without reference to any content of 
the image sequence, and is performed without reference to any three dimensional real 
world space locations or dimensions (see Kreitman, col. 7, line 64 to col. 8, line 6, 
wherein the transforming step performed by transformer 64 comprises all applied 
geometric transformations of the visual content, is performed without reference to any 
content of the image sequence, and is performed without reference to any three 
dimensional real world space locations or dimensions). 

With regard to claim 2, Kreitman describes wherein the three dimensional real 
world space locations include a location of a camera which originally produced the 
image sequence (see Kreitman, col. 7, line 64 to col. 8, line 6, wherein the transforming 
step performed by transformer 64 is performed without reference to any three 
dimensional real world space locations or dimensions at all, including a location of a 
camera which originally produced the image sequence). 

With regard to claim 3, Kreitman describes wherein the three-dimensional real 
world scale dimensions comprise units of physical measurement relating to the scene of 
the image sequence (see Kreitman, col. 7, line 64 to col. 8, line 6, wherein the 
transforming step performed by transformer 64 is performed without reference to any 
three dimensional real world space locations or dimensions at all, including units of 
physical measurement relating to the scene of the image sequence). 

With regard to claim 4, Kreitman describes wherein the blending step is 
performed without reference to any three-dimensional real world space locations (see 
Kreitman, col. 6, lines 25-37 and col. 8, lines 1-9, wherein blending mask 58 is utilized 
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to blend the image 42 with the colors of the field on which the image 42 is implanted, 
without reference to any three-dimensional real world space locations). 

With regard to claim 5, Kreitman describes wherein the target location may 
change relative to the scene (see Kreitman, Fig. 3, possible target locations 52, col. 6, 
lines 17-24, wherein the implantation designer designs the image 42 to be implanted 
and determines where on the model 50 to place it at one of the plurality of target 
positions 52) and as the image sequence progresses (see Kreitman, col. 1, lines 19-28, 
wherein target locations are re-selected as the image sequence progresses). 

With regard to claim 6, Kreitman describes wherein the image sequence is 
received from an analog video recorder, a digital video recorder or a broadcast quality 
video camera (see Kreitman, Fig. 2, broadcast quality video cameras 30 and col. 5, 
lines 22-36, wherein the image sequence is inherently received from an analog video 
recorder since videotape is can be a source of the image sequence). 

With regard to claim 18, Kreitman describes applying a transform function to a 
marker line that is to be inserted into an image sequence that was produced by a 
camera (see Kreitman, Figs. 2, 4, and col. 7, line 64 to col. 8, line 9, wherein image 42 
of Fig. 4 which is comprised of marker lines is transformed to match the perspective of 
the input video frame of Fig. 2 and is inserted into the video frame of Fig. 2); and 
blending the transformed line with the image sequence (see Kreitman, col. 6, lines 26- 
37, Fig. 4C, and col. 8, lines 1-9, wherein transformed blending mask 58 is utilized to 
blend the image 42 with the colors of the field on which the image 42 is implanted); 
wherein the transforming step and the blending step together result in insertion of the 
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line into the image sequence such that the line appears at a target location as if it had 
been part of the original scene displayed by the image sequence (see Kreitman, col. 8, 
lines 1-9, wherein if players walk on the part of the court 32 where the image 42 is 
implanted, they will appear to walk "over" the implanted image); and wherein the 
transforming step comprises all applied geometric transformations of the line, is 
performed without reference to any real-time content of the image sequence, and is 
performed without reference to any location of the camera (see Kreitman, col. 7, line 64 
to col. 8, line 6, wherein the transforming step performed by transformer 64 comprises 
all applied geometric transformations of the visual content, is performed without 
reference to any content of the image sequence, and is performed without reference to 
any three dimensional real world space locations or dimensions). 

With regard to claim 22, Kreitman describes applying a transform function to a 
virtual sign that is to be placed into an image sequence that was produced by a camera 
(see Kreitman, Figs. 2, 4, and col. 7, line 64 to col. 8, line 9, wherein image 42 of Fig. 4 
is transformed to match the perspective of the input video frame of Fig. 2 and is inserted 
into the video frame of Fig. 2); and blending the transformed sign with the image 
sequence (see Kreitman, col. 6, lines 26-37, Fig. 4C, and col. 8, lines 1-9, wherein 
transformed blending mask 58 is utilized to blend the image 42 with the colors of the 
field on which the image 42 is implanted); wherein the transforming step and the 
blending step together result in insertion of the sign into the image sequence such that 
the sign appears at a target location as if it had been part of the original scene displayed 
by the image sequence (see Kreitman, col. 8, lines 1-9, wherein if players walk on the 
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part of the court 32 where the image 42 is implanted, they will appear to walk "over" the 
implanted image); and wherein the transforming step comprises all applied geometric 
transformations of the sign, is performed without reference to any content of the image 
sequence, and is performed without reference to location of the camera (see Kreitman, 
col. 7, line 64 to col. 8, line 6, wherein the transforming step performed by transformer 
64 comprises all applied geometric transformations of the visual content, is performed 
without reference to any content of the image sequence, and is performed without 
reference to any three dimensional real world space locations or dimensions). 

With regard to claim 26, Kreitman describes applying a transform function to 
visual content to be inserted into an image sequence, wherein the image sequence 
includes a moving object (see Kreitman, Figs. 2, 4, and col. 7, line 64 to col. 8, line 9, 
wherein image 42 of Fig. 4 is transformed to match the perspective of the input video 
frame of Fig. 2 and is inserted into the video frame of Fig. 2, and wherein the input video 
frame includes moving objects such as tennis players 44 of Fig. 6); and blending the 
transformed visual content with the image sequence (see Kreitman, col. 6, lines 26-37, 
Fig. 4C, and col. 8, lines 1-9, wherein transformed blending mask 58 is utilized to blend 
the image 42 with the colors of the field on which the image 42 is implanted); wherein 
the transforming step and the blending step together result in insertion of the content 
into the image sequence such that the content appears at a target location as if it had 
been part of the original scene displayed by the image sequence, and such that the 
content is located on the moving object as the object moves in the scene (see Kreitman, 
col. 8, lines 1-9, wherein if the tennis players walk on the part of the court 32 where the 
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image 42 is implanted, they will appear to walk "over" the implanted image); and 
wherein the transforming step comprises all applied geometric transformations of the 
visual content, is performed without reference to any content of the image sequence, 
and is performed without reference to any three dimensional real world space locations 
or dimensions (see Kreitman, coL 7, line 64 to col. 8, line 6, wherein the transforming 
step performed by transformer 64 comprises all applied geometric transformations of 
the visual content, is performed without reference to any content of the image 
sequence, and is performed without reference to any three dimensional real world 
space locations or dimensions). 

With regard to claim 34, Kreitman describes applying a transform function to 
transform a first point P[A] within a first image to a second point P[B] within a second 
image (see Kreitman, Figs. 2, 4, and col. 7, line 64 to col. 8, line 9, wherein image 42 of 
Fig. 4 is transformed to match the perspective of the input video frame of Fig. 2 and is 
inserted into the video frame of Fig. 2); wherein the first image contains content related 
to a first camera view (see Kreitman, Fig. 3 and col. 6, lines 7-16, wherein image 42 is 
content related to the computer's inherent virtual camera "top view" of geometric tennis 
court model 50); wherein the second image contains content related to a second 
camera view (see col. 7, lines 30-67, wherein the input video frame is generated by 
active camera 30); wherein the point P[A] corresponds to the same location within real 
world space as point P[B] (see Kreitman, col. 6, lines 7-16, wherein the geometric tennis 
court model 50 is a scaled version of the real tennis court 32, therefore, points on the 
two tennis courts correspond); wherein the transforming step is performed without 
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reference to any content of the first image and the second image; wherein the 
transforming step is further performed without reference to any three-dimensional real 
world space locations or dimensions as related to the content of the first image and the 
second image (see Kreitman, col. 7, line 64 to col. 8, line 6, wherein the transforming 
step performed by transformer 64 comprises all applied geometric transformations of 
the visual content, is performed without reference to any content of the image 
sequence, and is performed without reference to any three dimensional real world 
space locations or dimensions). 

With regard to claim 35, Kreitman describes a set of fixed transform parameters 
(see Kreitman, col. 7, lines 58-63, generally the features of the playing field, specifically, 
tennis court lines 38 which are fixed and which are used to produce a transform matrix); 
first sensor data received from a first camera corresponding to the first camera view 
(col. 7, line 64 to col. 8, line 9, image 42, wherein image 42 is sensed data related to the 
computer's virtual camera "top view" of geometric tennis court model 50 of Fig. 3); 
second sensor data received from a second camera corresponding to the second 
camera view (col. 7, line 64 to col. 8, line 9, the input video frame captured by cameras 
30 as shown in Fig. 2); a set of dynamic transform parameters that are based on the 
fixed transform parameters, the first sensor data and the second sensor data (see 
Kreitman, col. 8, lines 1-9, wherein transformed blending mask 58 is a dynamic 
parameter because it is utilized to blend the image 42 with the dynamic colors of the 
video image on which the image 42 is implanted). 
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With regard to claim 36, Kreitman describes pan information; tilt information; 
zoom information; and focus information (see Kreitman col. 3, lines 33-39, wherein pan 
information is the same as turn information turn information, and wherein focus 
information is inherent for a camera). 



Claim Rejections • 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 30 is rejected under 35 U.S.C. 103(a) as being unpatentable over 

Kreitman, U.S. Patent No. 5,491,517, in view of Rosser, U.S. Patent No. 6,750,919. 

With regard to claim 30, Kreitman describes applying a transform function to 

visual content (see Kreitman, Figs. 2, 4, and col. 7, line 64 to col. 8, line 9, wherein 

image 42 of Fig. 4 is transformed to match the perspective of the input video frame of 

Fig. 2 and is inserted into the video frame of Fig. 2); and blending the transformed visual 

content with an image sequence (see Kreitman, col. 6, lines 26-37, Fig. 4C, and col. 8, 

lines 1-9, wherein transformed blending mask 58 is utilized to blend the image 42 with 

the colors of the field on which the image 42 is implanted); wherein the transforming 

step and the blending step together result in insertion of the content into the image 

sequence such that the visual content appears as if it had been part of the original 

scene displayed by the image sequence (see Kreitman, col. 8, lines 1-9, wherein if 
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players walk on the part of the court 32 where the image 42 is implanted, they will 
appear to walk "over" the implanted image); wherein said transforming step comprises 
all applied geometric transformations of the visual content, is performed without 
reference to any real-time image content of the image sequence and is performed 
without reference to any three dimensional real world space locations as related to the 
image sequence (see Kreitman, col. 7, line 64 to col. 8, line 6, wherein the transforming 
step performed by transformer 64 comprises all applied geometric transformations of 
the visual content, is performed without reference to any content of the image 
sequence, and is performed without reference to any three dimensional real world 
space locations or dimensions). Kreitman fails to explicitly describe dynamically 
changing visual content, as required by claim 30. However, DiCicco teaches 
dynamically changing visual content (see Title and Abstract of DiCicco). 

It would have been obvious to one of ordinary skill in the art at the time of 
invention by applicant to modify Kreitman to incorporate the dynamically changing visual 
content of Rosser, because Kreitman uses advertising as its visual content, and 
dynamically changing visual content is more likely to be attractive to the viewer, and is 
more likely to get the viewer's attention. 

Allowable Subject Matter 

Claims 7-17, 19-21, 23-25, 27-29, 31-33, and 37-38 are objected to as being 
dependent upon a rejected base claim, but would be allowable if rewritten in 
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independent form including all of the limitations of the base claim and any intervening 
claims. 

With regard to claim 7, the claim language "referencing a dynamic scene 
component model set which contains a three dimensional model for each of a subset of 
objects present in the scene that is displayed by the image sequence; receiving camera 
sensor data corresponding to the input image; selecting one scene component model 
out of the scene component model set; and filling a graphics frame buffer (GFB) with 
updated pixel data; and wherein the transforming step comprises: transforming the 
selected scene component model such that, when inserted into the input image, the 
selected scene component model will appear as if it had been part of the original scene 
displayed by the input image, thus creating an input embedded model; and texturing the 
input embedded model with said GFB, thus creating an input embedded GFB model; 
and wherein the blending step comprises blending the input embedded GFB model with 
the input image such that the input embedded GFB model appears as if it had been part 
of the original scene displayed by the input image, thus creating a blended image; and 
wherein the method also comprises the step of transmitting the input image or the 
blended image according to a selection of a user", in combination with the claim 
limitations of its respective parent claim(s), contains allowable subject matter. 

With regard to claim 19, the claim language "wherein an instance of the marker 
line is placed into an input image within the image sequence; further comprising the 
steps of: referencing a dynamic scene component model set which contains a three 
dimensional model for each of a subset of objects present in the physical scene that is 
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displayed by the image sequence; receiving camera sensor data corresponding to the 
input image; selecting one scene component model out of said scene component model 
set; and filling a graphics frame buffer (GFB) with updated pixel data; and wherein the 
transforming step comprises: transforming the selected scene component model such 
that, when inserted into the input image, the selected scene component model will 
appear as if it had been part of the original scene as displayed by the input image, thus 
creating an input embedded model; and texturing the input embedded model with said 
GFB, thus creating an input embedded GFB model; and wherein the blending step 
comprises blending the input embedded GFB model with the input image such that the 
input embedded GFB model appears as if it had been part of the original scene 
displayed by the input image, thus creating a blended image; and wherein the method 
also comprises the step of transmitting the input image or the blended image according 
to a selection of a user", in combination with the claim limitations of its respective parent 
claim(s), contains allowable subject matter. 

With regard to claim 23, the claim language "wherein an instance of the sign is 
placed into an input image within the image sequence; further comprising the steps of: 
referencing a dynamic scene component model set which contains a three dimensional 
model for each of a subset of objects present in the physical scene that is displayed by 
the image sequence; receiving camera sensor data corresponding to the input image; 
selecting one scene component model out of the scene component model set; and 
filling a graphics frame buffer (GFB) with updated pixel data; and wherein the 
transforming step comprises: transforming the selected scene component model such 
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that, when inserted into the input image, the elected scene component model will 
appear as if it had been part of the original scene as displayed by the input image, thus 
creating an input embedded model; and texturing the input embedded model with the 
GFB, thus creating an input embedded GFB model; and wherein the blending step 
comprises blending the input embedded GFB model with the input image such the said 
input embedded GFB model appears as if it had been part of the original scene 
displayed by the input image, thus creating a blended image; and wherein the method 
also comprises the step of transmitting the input image or the blended image according 
to a selection of a user", in combination with the claim limitations of its respective parent 
claim(s), contains allowable subject matter. 

With regard to claim 27, the claim language "wherein a portion of the content is 
placed into an input image within the image sequence; comprising the additional steps 
of: referencing a dynamic scene component model set which contains a three 
dimensional model for each of a subset of objects present in the scene that is displayed 
by the image sequence; receiving camera sensor data corresponding to the image; 
selecting one scene component model out of the scene component model set; and 
filling a graphics frame buffer (GFB) with updated pixel data; and wherein the 
transforming step comprises: transforming the selected scene component model such 
that, when inserted into the input image, the selected scene component will appear as if 
the selected scene component model had been part of the original scene displayed by 
the input image, thus creating an input embedded model; and texturing the input 
embedded model with said GFB, thus creating an input embedded GFB model; and 
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wherein the blending step comprises blending the input embedded GFB model with the 
input image such that the input embedded GFB model appears on the moving object as 
if the input embedded GFB model had been part of the original scene displayed by the 
input image thus creating a blended image; and wherein the method also comprises the 
step of transmitting either the input image or the blended image, according to a 
selection of a user", in combination with the claim limitations of its respective parent 
claim(s), contains allowable subject matter. 

With regard to claim 31, the claim language "wherein a portion of the dynamically 
changing visual content is placed into an input image within the image sequence; and 
comprising the additional steps of: referencing a dynamic scene component model set 
which contains a three dimensional model for each of a subset of objects present in the 
scene that is displayed by the image sequence; receiving camera sensor data 
corresponding to the input image; selecting one scene component model out of the 
scene component model set; and filling a graphics frame buffer (GFB) with updated 
pixel data; and wherein the transforming step comprises: transforming the selected 
scene component model such that, when inserted into the input image, the selected 
scene will appear as if it had been part of the original scene displayed by the input 
image, thus creating an input embedded model; and texturing the input embedded 
model with said GFB, thus creating an input embedded GFB model; and wherein the 
blending step comprises blending the input embedded GFB model with the input image 
such that the input embedded GFB model appears as if the input embedded GFB model 
had been part of the original scene displayed by the input image, thus creating a 
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blended image; and wherein the method also comprises the step of transmitting either 
the input image or the blended image, according to a selection of a user", in 
combination with the claim limitations of its respective parent claim(s), contains 
allowable subject matter. 

With regard to claim 37, the claim language "receiving an image sequence from a 
live camera that is outfitted with camera sensors; receiving a stream of camera sensor 
data from the attached camera sensors, wherein the stream of camera sensor data 
comprises camera sensor information applicable to images within the image sequence; 
receiving selections from a user, via direct input on one or more of the images, of 
multiple points of interest within 2D camera view space, wherein the selections have 
been received at multiple levels of camera zoom and focus; correlating, in time, each of 
the images for which selections were made and each of the user selections to the 
stream of camera sensor data; determining, based on the user selections and the 
camera sensor data, a set of zoom/focus transform parameters to be included within the 
set of fixed transform parameters; receiving selections from a user, via direct input on 
one or more images, of multiple points of interest within two-dimensional camera view 
space; wherein the selections have been received at multiple levels of camera pan and 
camera tilt; correlating, in time, each of the images for which selections were made and 
each of the user selections to the first and second camera sensor data; and 
determining, based on the user selections, and the camera sensor data, a set of other 
parameters to be included within the set of fixed transform parameters", in combination 
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with the claim limitations of its respective parent claim(s), contains allowable subject 
matter. 



Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jon Hadidi whose telephone number is 703-605-1 187. 
The examiner can normally be reached on M-F 8:00-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi can be reached on 703-305-4713. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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